Keyframe Sampling, Optimization, and Behavior Integration: A New Longest Kick in the RoboCup 3D Simulation League

Even with improvements in machine learning enabling robots to quickly optimize and perfect their skills, developing a seed skill from which to begin an optimization remains a necessary challenge for large action spaces. This thesis proposes a method for creating and using such a seed by i) observing the effects of the actions of another robot, ii) further optimizing the skill starting from this seed, and iii) em- bedding the optimized skill in a full behavior. Called KSOBI, this method is fully implemented and tested in the complex RoboCup 3D simulation domain. The main result is a kick that, to the best of our knowledge, kicks the ball farther in this simulator than has been previously documented.Computer Science

UT Austin Villa 2014: RoboCup 3D Simulation League Champion via Overlapping Layered Learning

Layered learning is a hierarchical machine learning paradigm that enables learning of complex behaviors by incrementally learning a series of sub-behaviors. A key feature of layered learning is that higher layers directly depend on the learned lower layers. In its original formulation, lower layers were frozen prior to learning higher layers. This paper considers an extension to the paradigm that allows learning certain behaviors independently, and then later stitching them together by learning at the "seams" where their influences overlap. The UT Austin Villa 2014 RoboCup 3D simulation team, using such overlapping layered learning, learned a total of 19 layered behaviors for a simulated soccer-playing robot, organized both in series and in parallel. To the best of our knowledge this is more than three times the number of layered behaviors in any prior layered learning system. Furthermore, the complete learning process is repeated on four different robot body types, showcasing its generality as a paradigm for efficient behavior learning. The resulting team won the RoboCup 2014 championship with an undefeated record, scoring 52 goals and conceding none. This paper includes a detailed experimental analysis of the team's performance and the overlapping layered learning approach that led to its success

Approximately Orchestrated Routing and Transportation Analyzer: Large-scale Traffic Simulation for Autonomous Vehicles

Abstract — Autonomous vehicles have seen great advancements in recent years, and such vehicles are now closer than ever to being commercially available. The advent of driverless cars provides opportunities for optimizing traffic in ways not possible before. This paper introduces an open source multiagent microscopic traffic simulator called AORTA, which stands for Approximately Orchestrated Routing and Transportation Analyzer, designed for optimizing autonomous traffic at a city-wide scale. AORTA creates scale simulations of the real world by generating maps using publicly available road data from OpenStreetMap (OSM). This allows simulations to be set up through AORTA for a desired region anywhere in the world in a matter of minutes. AORTA allows for traffic optimization by creating intelligent behaviors for individual driver agents and intersection policies to be followed by these agents. These behaviors and policies define how agents interact with one another, control when they cross intersections, and route agents to their destination. This paper demonstrates a simple application using AORTA through an experiment testing intersection policies at a city-wide scale. I